Multi-Modal Sentiment Analysis Based on Interactive Attention Mechanism

In recent years, multi-modal sentiment analysis has become more and popular in the field of natural language processing. Multi-modal mainly concentrates on text, image audio information. Previous work based BERT utilizes only text representation to fine-tune BERT, while ignoring importance nonverbal Most current research methods are fine-tuning models that do not optimize BERT’s internal structure. Therefore, this paper, we propose an optimized model is composed three modules: Hierarchical Multi-head Self Attention module realizes hierarchical extraction process features; Gate Channel replaces original Feed-Forward layer realize information filtering; tensor fusion self-attention mechanism utilized implement different modal features. CMU-MOSI, a public mult-imodal dataset, accuracy F1-Score were improved by 0.44% 0.46% compared with using custom fusion. Compared traditional models, such as LSTM Transformer, they certain extent.